How Well Have Morningstar Ratings Sorted Global Funds?

We recently conducted a study analyzing the performance of Morningstar ratings assigned to funds available for sale in markets outside the United States, such as Europe, Asia, and Australia. Those ratings systems include the Morningstar Rating for funds (that is, the “star rating”), the Morningstar Analyst Rating, and the Morningstar Quantitative Rating for funds.

We found that, on balance, these ratings have done a good job of sorting global funds based on future performance. Higher-rated funds were more likely to survive and outperform over subsequent horizons, while lower-rated funds were likelier to lag or go out of business. While this is encouraging, there are also opportunities for improvement.

An Overview of the Fund Ratings

Below is a table that summarizes the key attributes of the four fund rating systems that we evaluated.

While the ratings share similarities, there are differences to note. The star rating is backward-looking, whereas the Analyst Rating and Quantitative Rating are forward-looking; the star rating and Quantitative Rating are quantitative in nature, while the Analyst Rating turns on analysts’ qualitative judgments.

Nevertheless, we know investors tend to use these rating systems to achieve better financial outcomes. Accordingly, we assessed whether the ratings tended to sort funds based on their future performance.

How We Analyzed Ratings Performance

We measured ratings performance in a few ways. We examined the trailing returns of all global funds and exchange-traded funds (excluding those registered in the U.S.) in each ratings cohort through Sept. 30, 2022. We also ran an “event study” in which we took monthly snapshots of the ratings since their inception, tracked performance of the rated funds over a subsequent event horizon, and then averaged the event horizon returns.

To control for stylistic differences, we compared rated funds’ returns to a relevant peer group average or to the benchmark index that corresponds to the category to which the funds had been assigned.

The study utilized rated funds’ net-of-fee returns (translated back to the U.S. dollar), excluding multiple share classes, and included dead funds whenever possible.

What We Found

In general, we found that the star rating, Morningstar Analyst Rating, and Morningstar Quantitative Rating were effective in sorting funds based on their future performance. Higher-rated funds were likelier to survive and outperform than lower-rated funds, though this was more pronounced when measured against the category average than against the costless benchmark index.

We review each rating system’s performance in the sections below.

Morningstar Rating (“star rating”): In our analysis, we found that the star rating effectively sorted funds based on their future performance, with higher-rated funds far likelier to survive and outperform average peers than lower-rated funds. That’s evident in the chart below, which shows the trailing average annual excess returns of funds in each ratings cohort versus their average peer through Sept. 30, 2022.

In summary, though the star rating has struggled recently, it did a good job of sorting funds over the trailing three-, five-, and 10-year periods.

We also examined the performance of the star ratings using the event-study procedure. This technique is less sensitive to starting and end point than trailing returns, as it takes monthly snapshots of rated funds, measures their performance over a subsequent event horizon (12, 36, or 60 months), and averages those measurements to give a fuller picture, as shown below.

Under the event-study approach, it is clearer that the star rating has been effective at sorting funds. Higher-rated funds notched higher average excess returns than their typical peers compared with how lower-rated funds did, on average.

Morningstar Analyst Rating: We also found that Analyst Ratings have effectively sorted funds based on future performance. Higher-rated funds were likelier to survive and outperform, while lower-rated funds were more likely to lag or die, though this effect was less strong when measured against the category index.

In evaluating the Analyst Rating, it is worth keeping in mind that we assign Analyst Ratings to funds that Morningstar’s manager research analysts cover. Our research has found that covered funds tend to be larger by assets, possess longer track records, and boast lower expenses, on average, when compared with funds not under analyst coverage. They have also been higher-performing, on average.

This is evident when we examine the average annual excess returns of analyst-rated funds using the event-study method through Sept. 30, 2022. We found the Analyst Ratings generally did a good job of sorting funds based on future performance, with higher-rated funds tending to outperform their average peer by a larger margin than lower-rated funds. But even lower-rated funds outperformed, partly reflecting the fact that covered funds are cheaper and higher performing, on average.

Although the magnitude differed, this pattern held when we measured the performance of rated funds against their assigned category benchmark index, as shown below.

Morningstar Quantitative Rating: The Morningstar Quantitative Rating for funds uses algorithmic techniques to infer the rating a manager research analyst would assign to a fund if an analyst covered it. It does so by studying relationships between analysts’ ratings-assignment tendencies and the characteristics of the funds they’re assigning those ratings to. By applying those lessons to the universe of funds that analysts don’t cover, the Morningstar Quantitative Rating can assign ratings in a way that emulates an analyst.

(Note that funds are eligible for an Analyst Rating or a Quantitative Rating, but not both. We assign Analyst Ratings to funds that manager research analysts cover and Quantitative Ratings to those they don’t cover.)

Although the Quantitative Rating is still relatively new, having launched only in 2017 outside the U.S., the early results have been encouraging. It has done a good job of sorting funds based on their future performance, as measured by their average event-horizon excess returns versus the category average.

Though performance has been weaker recently, this pattern held even when we tallied funds’ average excess returns versus their assigned benchmark index, with higher-rated funds lagging by less than lower-rated funds and Gold-rated funds slightly topping their index over the average 36-month horizon following assignment of the rating.

Opportunities for Improvement

These findings are broadly encouraging, but we’d be remiss if we didn’t also address some opportunities for improvement:

• The star rating doesn’t “corner” well. While the star rating has been effective in sorting funds based on future performance over the long haul, it has struggled lately, with lower-rated funds outperforming higher-rated funds. The star rating tends to sort the best when markets are trending higher but has been prone to falter amid transitions, when market leadership changes. This is a limitation of a rating system that is based entirely on past performance, explaining why we consider the star rating a starting point for research, not an all-in selection tool.
• Morningstar Medalists haven’t beaten their benchmarks. The forward-looking Analyst Rating and Quantitative Rating have generally been successful sorting funds based on future performance versus a category average, with higher-rated funds succeeding more often than lower-rated funds, on average. Nevertheless, funds with Morningstar Analyst Ratings of Gold, Silver, and Bronze have not outperformed their assigned benchmark indexes, on average. This leaves room for improvement, as these ratings systems aspire not just to sort well based on future performance but to point investors toward funds that outperform their indexes over a full market cycle. So far, they haven’t met that mark.